A Structure from Motion Approach using Constrained Deformable Models and Appearance Prediction

In this technical report, we address the problem of recovering 3-D models from sequences of uncalibrated images with unknown correspondence. To that end, we integrate tracking, structure from motion with geometric constraints, and use of deformable 3-D models in a single framework. The key to making the proposed approach work is the use of appearance-based model matching and refinement. This appearance-based constrained structure from motion (AbCSfm) approach is especially useful in recovering shapes of objects whose general structure is known but which may have little discernable texture in significant parts of their surfaces. We applied the proposed approach to 3D face modeling from multiple images to create new 3-D faces for DECface, a synthetic talking head developed at Cambridge Research Laboratory, Digital Equipment Corporation. The DECface model comprises a collection of 3-D triangular and rectangular facets, with nodes as vertices. In recovering the DECface model, we assume that the sequence of images is taken with a camera with unknown camera focal length and extrinsic parameters (i.e., camera pose). Results of this approach show its good convergence properties and its robustness against cluttered backgrounds. c Digital Equipment Corporation, 1997 This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of the Cambridge Research Laboratory of Digital Equipment Corporation in Cambridge, Massachusetts; an acknowledgment of the authors and individual contributors to the work; and all applicable portions of the copyright notice. Copying, reproducing, or republishing for any other purpose shall require a license with payment of fee to the Cambridge Research Laboratory. All rights reserved. CRL Technical reports are available on the CRL’s web page at http://www.crl.research.digital.com. Digital Equipment Corporation Cambridge Research Laboratory One Kendall Square, Building 700 Cambridge, Massachusetts 02139